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(54) Parallel architecture for a high definition television video decoder having multiple 
independent frame memories 



(57) A parallel decoder for an MPEG-2 encoded 
video signal includes a deformatter which separates the 
input bit stream into multiple portions, each representing 
a respectively different section of the HDTV image. The 
separate portions are processed in parallel by respective 
decoders. In order to perform motion compensated 



processing, each of the four decoders includes a mem- 
ory that holds data representing the entire image. Each 
decoder provides its decoded output data to all of the 
decoders to maintain the data in the respective memo- 
ries. 
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Description 



BACKGROUND OF THE INVENTION 



5 The present invention relates to decoding of digitally encoded video signals and in particular to a decoder which 

decodes video signals that have been variable length encoded according to the MPEG-2 standard. 

BACKGROUND OF THE INVENTION 

10 In the United States a standard has been proposed for digitally encoded high definition television signals. This 
standard is essentially the same as the MPEG-2 standard, proposed by the Moving Picture Experts Group (MPEG) of 
the International Standards Organization (ISO). This standard is described in a draft internal standard (DIS) publication 
entitled "Information Technology - Generic Coding of Moving Pictures and Associated Audio, Recommendation H.626" 
I SO/I EC 1 381 8-2 DIS, 3/94 which is available from the ISO and which is hereby incorporated by reference for its teaching 

75 on the MPEG-2 digital video encoding standard. 

The MPEG-2 standard is actually several different standards. In MPEG-2 several different profiles are defined, each 
corresponding to a different level of complexity of the encoded image. For each profile, different levels are defined, each 
level corresponding to a different image resolution. One of the MPEG-2 "standards", known as Main Profile, Main Level 
is intended for encoding video signals conforming to existing television standards (i.e., NTSC and PAL). Another "stand- 

20 ard", known as Main Profile, High Level is intended for encoding high-definition television images. Images encoded 
according to the Main Profile, High Level standard may have as many as 1 ,152 active lines per image frame and 1 ,920 
pixels per line. 

The Main Profile, Main Level standard, on the other hand, defines a maximum picture size of 768 pixels per line 
and 567 lines per frame. At a frame rate of 30 frames per second, signals encoded according to this standard have a 

25 data rate of 1 3,063,680 pixels per second. By contrast, images encoded according to the Main Profile, High Level stand- 
ard have a maximum data rate of 1 ,152 * 1 ,920 * 30 or 66,355,200 pixels per second. This data rate is more than five 
times the data rate of image data encoded according to the Main Profile Main Level standard. The standard proposed 
for HDTV encoding in the United States is a subset of this standard, having as many as 1,080 lines per frame, 1 ,920 
pixels per line and a maximum frame rate, for this frame size, of 30 frames per second. The maximum data rate for this 

30 proposed standard is still far greater than the maximum data rate for the Main Profile, Main Level standard. 

Because of this difference in data rates, it is unlikely that the equipment used to decode Main Profile Main Level 
encoded data will be sufficient to decode high-definition television signals that have been encoded according to the Main 
Profile, High Level standard. 

To be optimally effective, the coding systems should be relatively inexpensive and yet have sufficient power to decode 

35 these digital signals in real time. 

The MPEG-2 standard defines a complex syntax which contains a mixture of data and control information. Some 
of this control information is used to enable the signals having several different formats to be covered by the standard. 
These formats define images, having differing numbers of picture elements (pixels) per line, differing numbers of lines 
per frame or field and differing numbers of frames or fields per second. In addition, the basic syntax of the MPEG-2 Main 

40 Profile defines the compressed MPEG-2 bit stream representing a sequence of images in six layers, the sequence layer, 
the group of pictures layer, the picture layer, the slice layer, the macro block layer, and the block layer. Each of these 
layers is introduced with control information. Finally, other control information, also known as side information, (e.g. frame 
type, macroblock pattern, image motion vectors, coefficient zig-zag patterns and dequantization information) are inter- 
spersed throughout the coded bit stream. 

45 To effectively receive the digital images, a decoder must recognize and extract the control portions and the side 
information, and then use the extracted information to process the video signal information. 

Using existing techniques, a decoder may be implemented using a single processor having a complex design and 
operating at a high data rate to perform this function. This high data rate, however, would require very expensive circuitry, 
which would be contrary to the implementation of a decoder in consumer television receiver in which cost is a major factor. 

50 

SUMMARY OF THE INVENTION 

The present invention is to be seen in the subject matter of claims 1, 6 and 10. In particular, the present invention 
is embodied in a decoder which is suitable for decoding high definition television images that have been encoded accord- 
55 ing to the MPEG-2 Main Profile, High Level standard. The exemplary decoder has a parallel processing architecture. 
This decoder includes a deformatter which separates the input bit stream into multiple portions, each representing a 
respectively different section of the HDTV image. The separate portions are processed in parallel by respective decoders. 
In order to perform motion compensated processing, each of the four decoders includes a memory that holds data 
representing the entire image. 
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According to one aspect of the invention, each decoder provides its output signal to all of the decoders to maintain 
the data in the respective memories. 

BRIEF DESCRIPTION OF THE DRAWINGS 

5 

Figure 1 is a block diagram of a high definition television decoder which includes an embodiment of the present 
invention. 

Figure 2 is a block diagram of a deformatter/router suitable for use with the high definition television decoder shown 
in Figure 1. 

10 Figure 3 is a timing diagram which is useful for describing the operation of the deformatter/router shown in Figure 2. 

Figure 4 is an image map diagram which is useful for describing the operation of the decoder shown in Figure 1. 

Figure 5 is a timing diagram which is useful for describing the operation of the decoder shown in Figure 1 . 

Figure 6 is a block diagram of circuitry suitable for use in the motion compensation processor shown in Figure 1 . 

Figure 7 is a frame timing diagram which is useful for describing the operation of the motion compensation processing 
is circuitry shown in Figure 6. 

DETAILED DESCRIPTION 

The description shows and explains in detail the present invention as described in the independent claims 1 , 6 and 

20 10. In addition, the features in the subclaims, which are also to be seen in connection with the independent claim 10, 
are explained hereinafter. The exemplary embodiment of the invention decodes HDTV signals which have been encoded 
according to the MPEG-2 standard and in particular, the Main Profile, High Level MPEG-2 standard. The invention 
described herein, however, is not limited to decoding this type of information. It may also be used to decode other types 
of variable-length encoded video data, where the encoded data corresponds to distinct areas of the original image. 

25 The MPEG-2 Main Profile standard defines a sequence of images in six levels: the sequence level, the group of 
pictures level, the picture level, the slice level, the macroblock level and the block level. Each of these levels may be 
considered to be a record in a data stream, with the later-listed levels occurring as nested sub-levels in the earlier listed 
levels. The records for each level, except the block level, include a header section which contains data that is used in 
decoding its sub-records. The actual image pixel data is contained in the block records. 

30 Each block of the encoded HDTV signal contains data representing 64 respective coefficient values of a discrete 
cosine transform (DCT) representation of 64 picture elements (pixels) in the HDTV image. 

In the encoding process, the pixel data is subject to motion compensated differential coding prior to the discrete 
cosine transformation and the blocks of transformed coefficients are further encoded by applying run-length and variable 
length encoding techniques. A decoder which recovers the image sequence from the data stream reverses the encoding 

35 process. This decoder employs a variable length decoder, an inverse discrete cosine transform processor and a motion 
compensation processor. 

Figure 1 is a block diagram of an exemplary HDTV decoding system, which may be used to decode high definition 
television signals that have been encoded according to the MPEG-2 standard. The diagram shown in Figure 1 does not 
address some aspects of the high definition television decoding standard, for example, the demodulation of the received 

40 radio frequency HDTV signals to recover the digital data stream. 

The decoder shown in Figure 1 , is arranged as four parallel decoders which are coupled to receive separate input 
data streams from a deformatter/router 110. These data streams are combined in an interpolation filter 1 1 8 and a raster 
converter 120 to produce video signals that are used to drive a high definition monitor (not shown). 

In the exemplary embodiment of the invention as shown in Figure 1 , the deformatter/router 110 receives a digital 

45 HDTV signal (DIN) as consecutive 8-bit bytes at its input port. The deformatter/router 1 10 performs two functions, it 
converts the 8-bit parallel bit stream into a bit serial data stream and it recognizes control information in the data stream 
down to the slice layer. The recognized control information is passed to all four of the decoders 1 1 la, 1 1 1b, 1 1 1c and 
1 1 1d. The slice start code and the data which follows it are passed to one of the four processors based on the horizontal 
starting position of the slice in the final image. 

so As described above, the MPEG-2 standard defines six layers of information for a sequence of images. These are 
the sequence layer, the group of pictures layer, the picture layer, the slice layer, the macroblock layer and the block layer. 

These various layers are defined in the above-referenced document which describes the MPEG-2 standard. Briefly, 
the sequence layer includes the other five layers. The layers may be referred to as layers of data records. Thus, a 
sequence record includes at least one group of pictures record which, in turn, includes at least one picture record and 

55 so on, down to the block layer. The sequence layer defines the starting point of a set of images which can be decoded 
based only on the information following the start of the sequence. The next lower layer is the group of pictures layer. 
This layer defines a subset of the images contained in the sequence. The images in the group of pictures layer generally 
include at least one intra-encoded image and several predictively or bidirectionally encoded images. 
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Immediately below the group of pictures layer is the picture layer. The data records at this layer define single frame 
images. Below the picture layer is the slice layer. Each record at this layer represents a single slice of the image repre- 
sented by the picture layer record. Below the slice layer are the macroblock and block layers. These layers represent 
successively more basic components of the image, as described above. 
5 Each of these level records begins with a start code. All sequence start codes begin with at least 23 zeros and a 

one.. The eight-bit value following the one determines the type of start code. Table 1 lists start codes which are relevant 
to the subject invention. 



Table 1 



10 


Start Code 


Hexadecimal Value 




PICTURE 


00000100 




SLICE (including vertical position) 


00000101 through 000001 AF 


15 


SEQUENCE 


000001 B3 




GROUP OF PICTURES 


000001 B8 



20 Following the sequence start code, are a plurality of parameters for the sequence of pictures. These include the 
horizontal and vertical size of the image in pixels, the pixel aspect ratio, the picture rate and the bit rate at which data is 
provided by the encoder. 

Another two items which may be included in the sequence header are quantization matrices to be used in decoding 
intra-f rame encoded data and non-intra-frame encoded data. If these matrices are not provided, default matrices, defined 

25 in the MPEG-2 standard, are used. 

After the header information in the sequence record are one or more group of pictures level records. Each group of 
pictures record includes a header, containing a time code and two flags which are used in decoding the following group 
of pictures. The pictures in the group are represented by several picture records included in the group of pictures record. 
Following the final group of pictures in the sequence record is a sequence end code. This code, which has the same 

30 general format as a start code, is defined as the hexadecimal value 000001 B7. 

Unless the format of the data transmission is well known, decoding may only begin after a sequence start code and 
its accompanying header parameters have been encountered in the data stream. To prevent long delays between 
sequence start codes, it is contemplated in the MPEG-2 standard that these start codes and their associated header 
information may be inserted multiple times in a given sequence. These header blocks may be inserted, for example, 

35 between alternate group of pictures records. 

The header for the picture layer includes a temporal reference, which defines the order in which the pictures are 
displayed; a picture coding type which indicates that the picture is either intra-encoded (l-frame) forward predictively 
encoded (P-frame) or bidirectionally predictively encoded (B-frame). The picture header also contains a value which 
indicates how much of the bit stream should be buffered before the decoding operation can begin, and, optionally, a pair 

40 of values which define how motion compensation processing is to occur on a whole-frame basis (e.g. to describe an 
underlying motion that affects the entire frame, such as an image pan). 

Following the header information for the picture record are one or more slice records. Each slice record includes 
one or more macroblock records. In ATVL's implementation, each slice defines a row of macroblocks which, when 
decoded, extend one-quarter of the way across the image. Figure 4 is a image map diagram of an HDTV image which 

45 shows slices 410a, 410b, 410c and 41 Od spanning the image horizontally. In the exemplary embodiment of the invention, 
each image includes 64 rows of slices, each row including four slices. 

Figure 1 is a block diagram of an HDTV decoder which includes an embodiment of the present invention. The 
decoder shown in Figure 1 is arranged in four parallel decoding channels. In general terms, a deformatter/router 110 
divides the demodulated bit stream into four parts, sending one part to each of the four channels, 1 1 1a, 1 1 1b, 1 1 1c and 

so 1 1 1d. As described below, each of these channels handles image data formatted in slices, representing respectively 
different sections of the resulting image, and produces decoded data in raster format. The decoded data provided by 
the four channels is horizontally interpolated in sections by the interpolation filter 118. The sections are then combined 
in the section raster to high-definition raster converter 120. 

Figure 2 is a block diagram of circuitry suitable for use as the deformatter/router 1 10. As shown in Figure 2, the 

55 deformatter/router includes an 8-bit parallel to serial converter 210 which converts the 8-bit parallel input signal into a 
bit-serial output signal, BSM. The signal BSM, in turn, is applied to a 32-bit shift register 212. The shift register 212 has 
a 32-bit parallel output port connected to a start code detector 214. The bit-serial signal provided by the shift register 
212 is applied in parallel to four gating circuits, 216a, 216b, 216c and 216d. The gating circuits are controlled by four 
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respective active-low control signals, ENa\ ENb\ ENc* and ENd'. These control signals are generated by the start code 
detector 214. 

In the exemplary embodiment of the invention, the start code detector 214 is a finite state machine which includes 
a memory (not shown) that receives the 32-bit signal provided by the shift register 212. As described above, the start 

5 codes appear in the bit-stream as 32-bit values: 23 zeros followed by a 1 and an eight-bit value which indicates the type 
of start code. The memory in the start code detector is programmed to recognize start codes for the sequence, group 
of pictures, picture and slice layers and the sequence stop code. 

When the detector 214 detects a sequence stop code or a start code that indicates a sequence, group-of-pictures 
or picture layer, it brings all of the enable signals, ENa\ ENb', ENc' and ENd' to a logic-low state, causing the gates 

10 216a, 216b, 216c and 216d to transmit the detected stop or start code and all of the information between the detected 
code and the next detected code to all four of the decoders 1 1 1a, 1 1 1b, 1 1 1c, and 1 1 1d. 

When the start code detector 214 encounters the first slice code in any vertical position, it sends the slice code and 
all information following that code (i.e. the slice record) only to decoder 111a. When the next slice code is encountered 
that slice record is sent to decoder 111b. The next following slice record is sent to decoder 111c, and the last slice record 

is at that vertical position is sent to decoder 111 d. As shown above in Table 1 , the vertical position of a slice is encoded 
in the low-order eight-bits of the slice start code. This position value is 1 at the top of the image and, under the MPEG- 
2 standard, may be as large as 175 (AF hexadecimal) at the bottom of the image. 

After processing four consecutive slice records, the next slice record encountered by the deformatter/router is sent 
to decoder 1 1 1a. The demultiplexing of the slice records continues in this manner until a start code other than a slice 

20 start code (including a sequence end code) is detected. This detected start code and the header information which 
follows it are broadcast to all four decoders as are all other such start codes and headers until a slice start code is 
encountered. This next slice start code marks the first slice record of a new picture which is sent to decoder 111a 
beginning the demultiplexing operation again. 

Figure 3 is a timing diagram which shows the operation of the deformatter/router. As shown in this Figure, the slice 

25 records for four consecutive slices are sent to respectively different ones of the decoders 111a, 111b, 111c and 1 1 1 d 
via the demultiplexed bit-stream signals BSMa, BSMb, BSMc and BSMd. The general record data (i.e. the header infor- 
mation from sequence records, group of pictures records and picture records) is sent to ail four of the decoders via all 
four of the bit stream signals BSMa, BSMb, BSMc and BSMd. 

As shown in Figure 4, using the exemplary encoding standard, four consecutive slices span the width of the HDTV 

30 image. Due to the demultiplexing performed by the deformatter/router 1 10, all slices in column 410a of the image 410 
are processed by decoder 111a, all slices in column 410b are processed by decoder 1 1 1b all slices in column 410c are 
processed by decoder 111c and all slices in column 41 Od are processed by decoder 111 d. 

Referring to Figure 1 , the demultiplexed bit stream provided by the deformatter/router 1 10 is applied to the variable 
length decoders 1 12a, 1 12b t 1 12c and 1 12d. Each of these decoders provides four output signals, a coefficient data 

35 signal, CD, and a control signal, IDCONT, that convey coefficient data and side information, respectively to one of the 
inverse discrete cosine transform (IDCT) processors 1 14a, 1 14b, 1 14c and 1 14d, a motion vector and side information 
data stream, MVS, and a horizontal blanking / vertical blanking signal, HB/VB, which is applied to a respective motion 
compensation processor 1 16a, 1 16b, 1 16c and 1 16d. 

Each variable length decoder 1 12 extracts header information from the general records that it receives and extracts 

40 and decodes run-length and code values for the variable length encoded DCT coefficient macroblocks that are applied 
to its input port. These run-length and code values are applied to a coefficient processor (not shown) internal to the 
variable length decoder 112, where they are converted into macroblocks of quantized coefficient values. 

The coefficients in each macroblock are dequantized in the IDCT processors 1 14a, 1 14b, 1 14c and 1 14d and the 
dequantized coefficients are then subject to an inverse discrete cosine transformation. The output signal, DP, provided 

45 by each of the IDCT processors is actually a pair of bit-streams, each bit-stream representing blocks of decoded pixel 
values that constitute the macroblocks. Figure 5 is a timing diagram which illustrates the format of the signals provided 
by each of the IDCT processors. 

As described above, each macroblock of pixel values includes four blocks of luminance information Y, and one block 
each of two color difference signals, U and V. In the exemplary embodiment of the invention, the luminance blocks are 

so numbered 0, 1, 2 and 3 and the chrominance blocks are numbered 4 and 5. The IDCT processors 114a, 114b, 114c 
and 1 1 4d provide the blocks of pixel values via two busses, B0 and B1 . Figure 5 illustrates the order in which the blocks 
that make-up a macroblock are provided via these two busses. 

Under the MPEG-2 standard, the macroblocks of pixel values may be either absolute pixel values or differential pixel 
values. When the macroblocks contain differential values, the corresponding macroblock also includes motion vector 

55 information. This motion vector information is decoded by the respective variable length decoders 11 2a, 1 1 2b, 1 1 2c and 
1 12d and is provided, via the signals MVS, to the respective motion compensation processors 116a, 116b, 116c and 
1 1 6d. The motion compensation processors, described in detail below with reference to Figures 6 and 7, use the absolute 
or differential pixel information and, optionally, the motion vector information to recover absolute pixel values for the 
image slices. These pixel values are provided by the motion compensation processor 1 16a to the interpolation filter 118. 
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Depending on the horizontal size of the input image, the interpolation filter 118 expands the image to fit the aspect 
ratio of the receiver's display device. In the exemplary embodiment of the invention, the luminance component, Y, of the 
image is only expanded horizontally by the interpolation filter 118. The chrominance components are expanded both 
horizontally and vertically by a factor of two to produce a number of chrominance samples of each of the U and V color 

5 difference signals which matches the number of samples of the interpolated luminance signal. It is contemplated, how- 
ever, that the interpolation filter 1 1 8 may be adapted to perform both horizontal and vertical interpolation on all of the 
image components. Y, U and V. 

The output signals of the interpolation filter 1 1 8 are applied to section-raster to high-definition-raster converter 1 20. 
Converter 120 combines the various interpolated section signals to produce samples representing the luminance signal, 

io Y and two color difference signals, U and V in proper raster format for display on the receiver's display device. In addition, 
the converter 120 generates a composite blanking signal for the receiver's display device. 

To understand the operation of the motion compensation processors 116a, 116b, 116c and 116d, it is helpful to 
understand the order in which individual picture records occur in a group of pictures record and the order in which the 
pictures are displayed. In the MPEG-2 standard, each group of pictures record generally includes one l-frame, several 

15 P-frames and several B-frames. The l-frame is provided to refresh the picture to compensate for errors which would 
otherwise propagate through a sequence of pictures. P-frames are frames that are predicted from an earlier l-frame or 
P-frame in the display sequence. B-frames are bidirectionally predicted from at least one of an earlier I- or P-frame and 
a later I- or P-frame in the display sequence. 

Because the B-frames are predicted from frames which occur later in the display sequence, the order of encoded 

20 frames in an MPEG-2 bit stream is not the same as the display order. One exemplary order is shown in Figure 7. As 
shown in this Figure, starting at time T3, the frames are received in the order l 0 , B_ 2l B_i, P3, Bi, B 2 , P6. B 4 , B5, Pg, B 7 , 
Be. When these frames are processed by the motion compensation processors 1 16, they are displayed in the order B„ 2 , 
B_i, lo, Bi, B 2 , P3, B4, B5, Pq, B7, Bg, P9. 

To accommodate both the forward and bidirectional prediction modes, each of the motion compensation processors 

25 1 16a, 1 16b, 1 16c and 1 16d includes two frame memories, one for holding a picture to be used in forward predictions 
and the other for holding a picture to be used in backward predictions. In addition, the motion compensation processors 
of the exemplary embodiment of the invention convert the block-format video information, in the input frame order, into 
raster format information for an interlace scan raster in the display frame order. As described below, each motion com- 
pensation processor 116 includes four half-frame memories for this purpose. 

30 Thus, even though each of the processors 116a through 116d processes only one segment of the image, as 
described above with reference to Figure 4, it stores, in the motion compensation memories 618a and 618b, two entire 
frames of video information. This architecture is desirable so that a given motion compensated macroblock of data in 
the segment that is being processed may obtain a motion reference macroblock from anywhere in the image. 

Figure 6 is a block diagram of circuitry suitable for use as the motion compensation processor 1 16a. The circuitry 

35 for the processors 1 16b, 1 16c and 1 16d may be identical. In Figure 6, the bus signals B0 and B1 provided by the IDCT 
processor 1 14a are applied to respective input ports of a first-in-first-out (FIFO) memory 610. The FIFO memory 610 
converts the stream of blocks of pixel data received from the IDCT processor 1 14a, as shown in Figure 5, into macrob- 
locks. The FIFO memory 610 provides one-half of the macroblock, representing the odd-numbered pixels of the mac- 
roblock, to one input port of an adder/clipper 612 and provides the other one-half of the macroblock, representing the 

40 even-numbered pixels of the macroblock to one input port of an adder/clipper 61 4. The other input ports of the adder/clip- 
pers 612 and 61 4 are coupled to receive corresponding odd-numbered and even-numbered pixels of a stored macroblock 
from a FIFO memory 624. The output ports of the adder/clippers 612 and 614 are combined such that a full macroblock 
of data is conveyed as the signal SECTION 0. 

In the exemplary embodiment of the invention, pixels from a prior stored frame (i.e. either an l-frame or a P-frame) 

45 are stored in one of the motion compensation memories 61 8a or 61 8b while pixels from a subsequent stored frame are 
stored in the other motion compensation memory 618b or 618a. The pixels stored into the memories 618a and 618b 
are received via four busses labeled SECTION 0 through SECTION 3. As shown in Figure 1 , the pixel values conveyed 
by each of these busses are generated by a respectively different one of the four motion compensation processors 1 1 6a, 
116b, 116c and 116d. In the exemplary embodiment of the invention, the SECTION 0 pixel values are produced by 

so processor 1 16a while the SECTION 1 values are produced by processor 1 16b and the SECTION 2 and SECTION 3 
values are produced by processors 1 16c and 1 16d, respectively. As described above, the SECTION 0 signal is provided 
as the combined output signal of the adder/clipper circuits 612 and 614. 

For data from l-frames, the output signal SECTION 0 is generated by passing the blocks provided via the busses 
B0 and B1 through the FIFO memory 61 0 and the adder/clippers 612 and 614 without modification. For P-and B-frames, 

55 the SECTION 0 signal is generated from the signals provided via the busses B0 and B1 and by motion compensated 
processing using data provided by the memories 618a and 618b. 

The data which is provided by the memories 61 8a and 61 8b is determined by the motion vector and side information 
signal, MVS, provided by the variable length code processor 1 12a. The signal MVS is provided to the motion compen- 
sated address generators 616a and 616b via the FIFO memory 61 1. In the exemplary embodiment of the invention, 
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either memory 61 8a or 61 8b may be used to hold the future frame or the past frame for motion compensation processing. 
Depending on which frames are stored in which memories, either of the address generators 616a or 616b may receive 
the forward motion vector data or backward motion vector data. Which frame memory 618a or 618b is used to hold 
which reference frame is determined by the datapath controller 626. Once an initial assignment has been determined, 
image frames stored in these memories are replaced by l-frame or P-frame data using a ping-pong method such that 
the oldest data is replaced by the newly received data. In addition, the datapath controller 626 determines when data 
is to be read from the FIFO's 610, 61 1 and 624 via the control signals CF610, CF61 1 and CF624, respectively. The 
controller 626 also receives the motion vector and side information signal MVS from the FIFO memory 61 1 . Based on 
these motion vectors, one or both of the motion compensation memories 618a and 618b provide data values to the 
respective half pixel generators 620a and 620b. 

In the MPEG-2 standard, it is permissible to classify motion, for the purpose of motion compensated processing, in 
one-half pixel intervals. Since only full pixel values are stored in the memories 618a and 618b, it is desirable to include 
some form of interpolation in an MPEG-2 decoder so that the full standard may be accommodated. In the exemplary 
embodiments of the invention, the half pixel generators 620a and 620b perform this interpolation function, responsive 
to control signals provided by the respective address generators 616a and 616b. 

When a B-frame is processed by the exemplary embodiment of the invention, the memories 618a and 618b and 
the half-pixel generators 620a and 620b each provide blocks of pixel values representing a macroblock at a displacement 
from the macroblock provided by the busses B0 and B1 as indicated by the motion vectors applied to the motion com- 
pensation address generators 616a and 616b. 

Even numbered pixels of each of the forward and backward macroblocks are applied to the multiplier 622a while 
odd numbered pixels of each macroblock are applied to the multiplier 622b. When a B-frame is being processed, each 
of the multipliers 622a and 622b combines respective pixel values from each of the received macroblocks in a set pro- 
portion and provides the combined pixel values to the FIFO memory 624. In the exemplary embodiment of the invention, 
the pixel values are combined in a 1:1 proportion. It is contemplated, however, that different proportions may be used 
and that the multipliers 622a and 622b may be programmed based on these provided proportion values, for example, 
by the motion compensation address generators 616a and 616b. responsive to side information conveyed as a part of 
the signal MVS. 

As described above, the FIFO memories 610 and 624 provide the odd-numbered pixels from the respective mac- 
roblocks to the adder/clipper 612 and provide the even-numbered pixels to the adder/clipper 614. These circuits sum 
the respective pixel values to generate the motion compensated pixel values. Each of the adder/clippers 614 also limits 
or clips the magnitude of the summed values to be less than or equal to 255. The odd-numbered and even-numbered 
pixels representing the motion compensated output pixel data are conveyed by separate busses (not shown) from the 
adder/clippers 612 and 614 to the memories 618a and 618b and the section selection circuitry 628. For convenience, 
these busses are shown in Figure 6 as being respective single busses. 

Each of the address generators 616a and 616b also receives a signal SMCMa and SMCMb, respectively, from a 
datapath controller 626. This signal indicates whether data provided via the SECTION 0 to SECTION 3 signals is to be 
written into the respective motion compensation memory 618a or 618b. As described above, these memories hold 
reference backward and forward prediction frames. Under the MPEG-2 standard, only an l-frame or a P-frame may be 
used as a reference prediction frame. Consequently, not every frame provided via the signals SECTION 0 through 
SECTION 3 is stored into one of the memories 618a and 618b. The signals SMCMa and SMCMb are selection signals 
which cause the reference frames to be stored into the memory 618a or 618b, respectively. The signals SMCMa and 
SMCMb are generated by a datapath controller 626, responsive to a frame type signal conveyed in the signal MVS 
provided by the variable length decoder 1 1 2a, shown in Figure 1 , via the FIFO memory 61 1 . If the received data is from 
an l-frame or a P-frame, the controller 626 activates the appropriate signal SMCMa or SMCMb to store the data into the 
memory 618a or 61 8b. The memory containing the oldest data is selected by the controller 626 to receive the new data. 
The datapath controller 626 also receives a macroblock synchronization signal, as a part of the signal MVS, which allows 
it to align the side information with the coefficient data. 

The odd-numbered and even-numbered pixels of each macroblock are provided via the signals SECTION 0, SEC- 
TION 1 , SECTION 2 and SECTION 3 to the section selection circuitry 628. The pixel values conveyed by each of these 
signals correspond to the section 0, section 1 , section 2 and section 3 divisions of the image frame shown in Figure 4. 
The section selection circuitry selects section data for storage into the raster memories 630a through 630d. 

In a raster scan image, data from one frame is displayed line-by-line from the top of the image to the bottom of the 
image. Because the data provided via the signals SECTION 0 through SECTION 3 are in macroblock order, the signals 
received via these lines are stored and reformatted using the section selection circuitry 628, the memories 630a, 630b, 
630c, and 630d, and the raster address generator 632. 

Each of the memories 630a, 630b, 630c, and 630d holds one-half of one frame of video information, representing 
successive ones of the sections shown in Figure 4. For example, since the circuitry shown in Figure 6 is circuitry for the 
motion compensation processor 1 16a shown in Figure 1 , each of the raster memories holds only that data which rep- 
resents section 0 and section 1 . The raster memories for processor 1 16b hold data for section 1 and 2, the memories 
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for processor 11 6c hold data for section 2 and section 3. The memories for processor 11 6d hold only that data which 
represents section 3. No additional section data is stored for this processor since none could be used by the interpolation 
filter to ensure that the right-hand edge of section 3 is properly interpolated. In the exemplary embodiment of the invention, 
the left-hand edge of section 0 and the right-hand edge of section 3 may not be properly interpolated. Because the pixel 
5 values for this data are displayed at the edges of the image, it is contemplated that they will be hidden in the overscan 
region of the display device. 

The memories 630a, 630b, 630c and 630d store image frames as they are provided via the signals SECTION 0 
and SECTION 1 and then provide the data in raster-scan order and in display frame order. The addresses and control 
signals for the memories 630a, 630b, 630c, and 630d are generated by the raster address generator 632. In generating 
10 these addresses, the generator 632 is responsive to a raster memory selection signal SRM provided by the datapath 
controller 626. 

The bus converter 634 demultiplexes the data stream, which is provided in the sample order Y.U.Y, V,Y,U, Y, V,... where 
Y represents a luminance sample and U and V represent samples of respective color difference signals. The output 
signals provided by the converter 634 are a luminance signal, Y, and a chrominance signal consisting of alternating U 
is and V samples. 

As shown in Figure 1, the motion compensation processor 116a receives information on horizontal and vertical 
blanking intervals from the VLD processor 112a and provides a horizontal blanking signal, HBO and vertical blanking 
signal VB. In the exemplary embodiment of the invention, this is accomplished via a signal HB/VB' applied to the bus 
converter 634 by the raster address generator 632. This signal is generated by raster address generator 632 based on 
20 the HB/VB signal received from VLD processor 1 1 2a. The exemplary generator 632 changes the timing of the horizontal 
and vertical blanking signals (HB/VB) to produce the signal HB/VB' which matches the data being read from the raster 
memories 630a through 630d. This changed signal is applied to the converter 634 which again changes timing of the 
horizontal and vertical blanking signals to produce the signals HBO and VB, which match the signals Y and U,V provided 
by the bus converter 634. 

25 Figure 7 is a timing diagram which illustrates how data values are written into and read from the various memories. 
The vertical divisions in Figure 7 represent various time intervals, To through T 15 . The first row, labeled Input, represents 
the order in which portions of the image frames are received at the motion compensation processor 1 1 6a via the busses 
B0 and B1 . Each of the six following rows describe when data is written into and read from the two motion compensation 
memories 618a and 618b and four raster memories 630a, 630b, 630c and 63 Od. The bottom row, labeled Output, 

30 describes the order in which frames are provided by the motion compensation processor 1 16a. 

Starting from time T3, a first l-frame, lo, is provided to the motion compensation processors 1 16 and is stored into 
one of the motion compensation memories, for example memory 618a and, at the same time, into, for example, the 
raster memory 630a. Since this is an intraframe coded image, no motion compensation processing is performed during 
the time interval T 3 . At time T 4 , segment data representing the frame B_ 2 are provided to the processor 116a and 

35 bidirectional motion compensation processing is performed on this frame using data from the frame P3, which was stored 
in motion compensation memory 61 8b at time To and data from the frame lo, stored in memory 61 8a. The result is stored 
into the raster memory 630c during time interval T 4 . Also during the time interval T 4 , segment data from the decoded 
frame P_3 is provided from the raster memory 630b. 

At time T5, data representing frame B_-| is received by the motion compensation processor 618a. This data is also 

40 subject to bidirectional motion compensation processing using the decoded frame lo stored in memory 618a and the 
processed frame P_3 stored in the memory 618b. The processed frame B^ is stored into the memory 630d during time 
interval T 5 while data from the processed frame B_ 2 is read from the memory 630c. 

During time interval T 6 , data from a predictive frame, P3 is applied to the motion compensation processor 618a. 
This data is processed using the data from the frame lo, which was stored into the motion compensation memory 618a 

45 during time interval T3. The resulting processed data for frame P3 is stored into the raster memory 630b during time 
interval T 6 . Also in time interval T 6 , the stored frame B_-j is provided from the memory 630d. At time T 7 , the bidirectional 
frame B^ is received by the processor 1 16a, it is processed using data in the stored frames lo and P3 and is stored into 
the raster memory 630c. During the interval T 7 , segment data from the stored frame lo is provided from the raster memory 
630a. 

so Processing continues in this manner with the frames B 2 , P6. B 4( B 5 , P 9 , B 7 and Bs being received during the respec- 
tive time intervals Tg. T9, T 10 , T<\<\, T 12 , T 13 and T 14 . During these time intervals, segment data from the processed image 
frames B 1f B 2 , P3, B 4 , B 5 , B 6 and B 7 are provided from the raster memories 630a through 630d as shown in Figure 7. 

It is noted that a processed image frame stored in one of the motion compensation memories 618a and 618b may 
be used, at different times, as both a forward and backward reference memory for bidirectional prediction. For example, 

55 during the time intervals T 4 and T 5 , the stored frame lo is used as a backward reference memory to predict values for 
the frames B_ 2 and B_i while during time intervals T6, T 7 and Ts it is used as a forward reference memory to predict 
pixel values for frames P 3 , B^ and B 2 . 

The invention has been described in terms of a system which uses four parallel channels, it is contemplated that it 
may be practiced by similar circuitry which divides the input bit-stream into differing numbers of channels or channels 
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which process different areas of the image. For example, it may be desirable to split the image vertically as well as 
horizontally such that, for example, the image is processed in four segments, each representing a respective quadrant 
of the image. 

While the invention has been described in terms of an exemplary embodiment it is contemplated that it may be 
practiced as outlined above within the spirit and scope of the appended claims. 

Claims 

1- A system suitable for decoding video signals, representing an image, that have been encoded according to the 
MPEG-2 standard, comprising: 

deformatting means for receiving the encoded video signals and for separating the received signals into N 
parallel bit streams, where N is an integer; 

N processing means, each coupled to the deformatting means to receive a respectively different one bit 
stream of the N parallel bit streams, each of the N processing means including variable length decoding means for 
decoding encoded image data and motion vector data from the one bit stream; and 

motion compensation processing means, coupled to the variable length decoding means for combining the 
decoded image data with data from a previously decoded image frame according to the motion vector data to produce 
motion compensated decoded image signals, wherein the motion compensation processing means includes a mem- 
ory which allows the motion compensation processing means to access an entire frame of the previously decoded 
image. 

2. A system according to claim 1 , wherein the motion compensation processing means further includes: 

means for sending the motion compensated decoded image signals to the motion compensation processing 
means of each of the N processing means; 

input means for receiving motion compensated decoded image signals from each of the N processing means; 

and 

means for storing the received motion compensated decoded image signals into the memory. 

3. A system according to claim 2, wherein the encoded video signal includes general data and data representing 
consecutive image slices and the deformatting means includes means for including the general data in all N of the 
parallel bit streams and for including data representing each consecutive slice in a respectively different one of the 
N bit-streams. 

4. A system according to claim 3, wherein the motion compensation processing means further includes a further frame 
memory for holding data from a further previously decoded image frame, wherein the means for storing the motion 
compensated decoded image signals into the memory further includes means for storing motion compensated 
decoded image signals representing the further previously decoded image frame into the further memory. 

5. A system according to claim 2, wherein each of the N processing means further includes: 

raster memory means for holding motion compensated image data representing a predetermined portion of 
the image; 

means coupled to the input means for storing the received motion compensated decoded image signals from 
M of the N processing means, as the predetermined portion of the image, into the raster memory means, where M 
is an integer less than N; and 

means coupled to the raster memory means for providing the stored image signals in raster scan order. 

6. Apparatus for decoding an encoded video signal representing a video image, which video signal has been encoded 
in segments corresponding to distinct regions of the image, the apparatus comprising: 

a deformatter which separates the encoded video signal into a plurality of signal portions, each representing 
a respectively different segment; and 

a plurality of decoders each coupled to receive a respectively different one of the plurality of signal portions 
provided by the deformatter, wherein each of the plurality of decoders includes: 

motion compensated processing means for decoding signal portions which have been encoded using motion 
compensated encoding techniques to produce decoded image data; and 

a memory which holds decoded data representing an entire previously received image and coupled to the 
motion compensated processing means to provide portions of the held data to the motion compensated processing 
means for use in decoding the encoded signal portions. 
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7. Apparatus according to claim 6, wherein each decoder further includes means for providing its decoded image data 
to the respective memories of each of the plurality of decoders to be stored as the held data. 

8. Apparatus according to claim 7, wherein: 

each of the plurality of decoders further includes: 

means for extracting motion vector data from the encoded video data; and 

means for providing the motion vector data to the motion compensated processing means; and 

the motion compensated processing means of each of the plurality of decoders further includes: 

a further memory which holds further decoded data representing a further entire previously received image; 

and 

means, responsive to the motion vector data for directing the decoded sections provided by the motion com- 
pensated processing means to one of the memory and the further memory. 

9. A system according to claim 8, wherein each of the plurality of decoders further includes: 

raster memory means for holding motion compensated image data representing a predetermined portion of 
the image; 

means coupled to the input means for storing the received motion compensated decoded image signals from 
M of the N processing means, as the predetermined portion of the image, into the raster memory means, where M 
is an integer less than N; and 

means coupled to the raster memory means for providing the stored image signals in raster scan order. 

10. A system suitable for decoding video signals, representing an image, that have been encoded according to the 
MPEG-2 standard, comprising: 

deformatting means for receiving the encoded video signals and for separating the received signals into N 
parallel bit streams, where N is an integer; 

N processing means, each coupled to the deformatting means to receive a respectively different one bit 
stream of the N parallel bit streams, each of the N processing means including: 

variable length decoding means for decoding encoded image data and motion vector data from the one bit 
stream; and 

motion compensation processing means, coupled to the variable length decoding means for combining the 
decoded image data with data from a previously decoded image frame according to the motion vector data to produce 
motion compensated decoded image signals, wherein the motion compensation processing means includes a mem- 
ory which holds an entire frame of the previously decoded image. 
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